Chiral spin density wave order on frustrated honeycomb and bilayer triangle lattice Hubbard model at half-filling

摘要

We study the Hubbard model on the frustrated honeycomb lattice withnearest-neighbor $t_1$ and second nearest-neighbor hopping $t_2$, which isisomorphic to the bilayer triangle lattice, using the SU(2)-invariant slaveboson theory. We show that the Coulomb interaction $U$ inducesantiferromagnetic (AF) chiral spin-density wave ($\chi$-SDW) order in a widerange of $\kappa =t_2/t_1$ where both the two-sublattice AF order at small$\kappa$ and the decoupled three-sublattice 120$^\circ$ order at large $\kappa$ are strongly frustrated, leading to three distinct phases with differentanomalous Hall responses. We find a continuous transition from a $\chi$-SDWsemimetal with anomalous Hall effect to a topological chiral Chern insulatorexhibiting quantum anomalous Hall effect, followed by a discontinuoustransition to a $\chi$-SDW insulator with zero total Chern number but anomalousac Hall effect.The $\chi$-SDW is likely a generic phase of strongly correlatedand highly frustrated hexagonal lattice electrons.